This invention relates to a method of identifying an object image from a camera image in order, for instance, to retrieve data on the object, in which a position of the object is mapped in a three-dimensional space by using the camera image of the object which is picked up by a television camera connected to a computer.
With respect to a mapping step for an object position in a camera image, the following two methods are known in the art which are described in a thesis entitled "Man-Machine Interface for Plant Control Centers to act Direct Manipulation Technique on Image" by Masayuki Tani et al. (IEE Paper(D), vol 111, No. 12, 1991, pp 1023 to 1030):
(1) First Mapping Method--A camera image display range is a two-dimensional orthogonal coordinates space, and the position of an object is given by two-dimensional orthogonal coordinates.
The first mapping method will be described with reference to FIGS. 7 and 8.
A left part of FIG. 7 shows an example of positional relationships between a television camera and an object. In the left part of FIG. 7, there is the television camera 1, a plane 2 is formed an image; and the object 3 is an image of which is to be picked up.
The right part of FIG. 7 shows a camera image display range in the arrangement shown in the left part. In the right part of FIG. 7, the camera image display range 4 and the image 5 of the object are represented (hereinafter referred to as "an object image", when applicable). The camera image display range 4 is a two-dimensional orthogonal coordinate space defined by coordinates range of (0, 0) to (Xmax, Ymax), and, in the camera image display range, the coordinates of the position of the object (the coordinates of the center of the object image 5) are (x1, y1).
A left part of FIG. 8 shows another example of positional relationships between the television camera and the object 3 in a case where a horizontal angle of the television camera 1 is changed. A right part of FIG. 8 shows a camera image display range 4 in the arrangement shown in the left part of FIG. 8. In FIG. 8, parts corresponding functionally to those which have been described with reference to FIG. 7 are therefore designated by the same reference numerals or characters; that is, in FIGS. 7 and 8, like parts are designated by like reference numerals or characters.
In FIG. 8, similarly as in FIG. 7, the image display range 4 is defined by the coordinates range (0,0) to (Xmax, Ymax). However, it should be noted that the coordinates of the position of the object are (x2, y1) different from those in the case of FIG. 7.
Hence, in the case of the first mapping method, as camera parameters such as the position, elevation angle, horizontal angle, and view angle of the television camera change, the coordinates of the position of the object (hereinafter referred to as "an object position", when applicable) are changed in the camera image display range. Therefore, in mapping the position of an object, the mapping operation must be carried out for each of the images picked up by the camera with the camera parameter changed.
(2) Second Mapping Method--A camera image is projected on a two-dimensional space in a three-dimensional space, and the position of the object and the camera are represented by three-dimensional orthogonal coordinates.
The second mapping method will be described with reference to FIGS. 9 and 10.
FIG. 9 shows one example of positional relationships between a position of the television camera, a position of the actual object 3, and a position of the object image 5. FIG. 10 shows another example of the positional relationships between the position of the television camera 1, the position of the actual object 3, and the position of the object images when the position and angles (horizontal angle, and elevation angle) of the camera are changed. In FIGS. 9 and 10, the object image 5 is represented in the image display range 4, and the remaining parts corresponding functionally to those which have been described with reference to FIGS. 7 and 8 are therefore designated by the same reference numerals or characters.
Further in FIG. 9, the position of the television camera in the three-dimensional orthogonal coordinate space is given coordinates (X1, Y1, Z1), the position of the actual object in the same space is given coordinates (X2, Y2, Z2), and the position of the object in the two-dimensional orthogonal coordinate space is given coordinates (X3, Y3) (which are the coordinates of the center of the object image 5). Similarly, in FIG. 10, the position of the television camera in the three-dimensional orthogonal coordinate space is given coordinates (X1', Y1', Z1'), the position of the actual object in the same space is given coordinates (X2, Y2, Z2), and the position of the object in the two-dimensional orthogonal coordinate space is given coordinates (X3', Y3') (which are the coordinates of the center of the object image 5).
In the second mapping method, the position of the actual object is represented by the coordinates in the three-dimensional orthogonal coordinate space as was described above. Hence, even when the parameters of the camera change, the coordinates of the position of the object in the plane of projection can be obtained by performing a projection calculation. However, in setting the object position coordinates, it is necessary to input three-dimensional coordinates data. In addition, whenever the parameters of the camera change, it is necessary to perform an intricate three-dimensional projection calculation.
Accordingly the above-described two mapping methods suffer from the following problems:
(1) First Mapping Method
When the camera parameters such as the position, elevation angle, horizontal angle, view angle of a camera change, the coordinates of the position of the object in the image display range are changed. Hence, as for each of the parameters of the camera, the coordinates must be determined. Thus, it is substantially difficult to apply the first mapping method to an image pickup system in which camera parameters may change.
(2) Second Mapping Method
The position of the camera, and the position of the actual object are given three-dimensional coordinates. Hence, the second mapping method is applicable to the case where the camera parameters change. However, whenever the camera parameters change, it is necessary to perform the three-dimensional projection calculation, and therefore the amount of three-dimensional projection calculation is considerably large. Furthermore, its data setting operation is rather troublesome, because it is necessary to provide three-dimensional coordinate data.